%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                               %
%                       Legal Notice                            %
%                                                               %
% This document is copyright (C) Olaf Kirch, okir@monad.swb.de  %
%                                                               %
% For copying conditions, please refer to the file COPYING.	%
%                                                               %
% The author is not liable for any damages, direct or indirect, %
% resulting from the use of information provided in this        %
% document.                                                     %
%                                                               %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% When viewing this document with vi on an 80-columns screen,   %
% do a ':se nonumber' first.                                    %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Definitions local to this chapter.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Warning!! These paths occur literally in a few verbatim
%% environments!!
%%
\def\newslib{\file{/usr/lib/news}}
\def\newsbin{\file{/usr/lib/news/bin}}
\def\newsspool{\file{/var/spool/news}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                               %
%               S T A R T   O F   C H A P T E R                 %
%                                                               %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{A Description of NNTP}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{nntp}
\index{configuring!NNTP|(}
\index{protocol!NNTP}
\index{news!nntpd@\prog{nntpd}}
\index{server!NNTP}
\index{NNTP}

\section{Introduction}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Due to the different network transport used, NNTP provides for a vastly
different approach to news exchange from C~news.  NNTP stands for
``Network News Transfer Protocol'', and is not a particular software
package, but an Internet Standard.\footnote{
  Formally specified in RFC~977.
} It is based on a stream-oriented connection~-- usually over TCP~--
between a client anywhere in the network, and a server on a host that keeps
netnews on disk storage. The stream connection allows the client and server
to interactively negotiate article transfer with nearly no turnaround
delay, thus keeping the number of duplicate articles low. Together with the
Internet's high transfer rates, this adds up to a news transport that
surpasses the original UUCP networks by far. While some years ago it was
not uncommon for an article to take two weeks or more before it arrived in
the last corner of Usenet, this is now often less than two days; on the
Internet itself, it is even within the range of minutes.


Various commands allow clients to retrieve, send and post articles.
The difference between sending and posting is that the latter may
involve articles with incomplete header information.\footnote{
  When posting an article over NNTP, the server always adds at least one
  header field, which is \mhdr{Nntp-Posting-Host:}. It contains the
  client's host name.
} Article retrieval may be used by news transfer clients as well as
newsreaders. This makes NNTP an excellent tool for providing news access to
many clients on a local network without going through the contortions that
are necessary when using NFS.


\index{news!pushing}
\index{news!pulling}
NNTP also provides for an active and a passive way of news transfer,
colloquially called ``pushing'' and ``pulling''. Pushing is basically
the same as the C~news ihave/sendme protocol. The client offers an
article to the server through the ``\keyword{IHAVE $<$var{msgid}$>$}''
command, and the server returns a response code that indicates whether
it already has the article, or if it wants it. If so, the client sends
the article, terminated by a single dot on a separate line.


Pushing news has the single disadvantage that it places a heavy load on
the server system, since it has to search its history database for every
single article.


The opposite technique is pulling news, in which the client requests a
list of all (available) articles from a group that have arrived after a
specified date. This query is performed by the \keyword{NEWNEWS}
command. From the returned list of message ids, the client selects those
articles it does not yet have, using the \keyword{ARTICLE} command for
each of them in turn.


The problem with pulling news is that it needs tight control by the server
over which groups and distributions it allows a client to request. For
example, it has to make sure that no confidential material from newsgroups
local to the site are sent to unauthorized clients.


There are also a number of convenience commands for newsreaders that
permit them to retrieve the article header and body separately, or even
single header lines from a range of articles. This lets you keep all
news on a central host, with all users on the (presumably local) network
using NNTP-based client programs for reading and posting. This is an
alternative to exporting the news directories via NFS which is described
in chapter~\ref{cnews}.


\index{news!faking}
An overall problem of NNTP is that it allows the knowledgeable to insert
articles into the news stream with false sender specification. This is
called \emph{news faking}.\footnote{
 The same problem exists with SMTP, the Simple Mail Transfer Protocol.
} An extension to NNTP allows to require a user authentication for
certain commands.


\index{nntpd@\prog{nntpd}}
\index{Lapsley, Phil}
\index{Barber, Stan}
There are a number of NNTP packages available. One of the more widely known
is the NNTP daemon, also known as the \emph{reference implementation}.
Originally, it was written by Stan Barber and Phil Lapsley to illustrate
the details of RFC~977.  Its most recent version is \prog{nntpd-1.5.11},
which will be described below.  You may either get the sources and compile
it yourself, or use the \prog{nntpd} from Fred van~Kempen's \file{net-std}
binary package.  No ready-to-go binaries of \prog{nntpd} are provided,
because of various site-specific values that must be compiled in.


The \prog{nntpd} package consists of a server and two clients for
pulling and pushing news, respectively, as well as an \prog{inews}
replacement. They live in a Bnews environment, but with a little
tweaking, they will be happy with C~news, too.  However if you plan to
use NNTP for more than offering newsreaders access to your news server,
the reference implementation is not really an option.  We will therefore
discuss only the NNTP daemon contained in the \prog{nntpd} package, and
leave out the client programs.


\index{Salz, Rich}
\index{INN}
There is also a package called ``InterNet News'', or INN for short, that was
written by Rich Salz.  It provides both NNTP and UUCP-based news
transport, and is more suitable for large news hubs.  When it comes to
news transport over NNTP, it is definitely better than \prog{nntpd}.
INN is currently at version \file{inn-1.4sec}.  There is a kit for
building INN on a \linux{} machine from Arjan de Vet; it is available
from \host{sunsite.unc.edu} in the \file{system/Mail} directory.  If you
want to set up INN, please refer to the documentation that comes along
with the source, as well as the INN~FAQ posted regularly to
\ngrp{news.software.b}.


\section{Installing the NNTP server}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{nntp.nntpd}

The NNTP server is called \prog{nntpd}, and may be compiled in two ways,
depending on the expected load on the news system.  There are no compiled
versions available, because of some site-specific defaults that are
hard-coded into the executable.  All configuration is done through
macro definines in \file{common/conf.h}.


\prog{nntpd} may be configured as either a standalone server that is
started at system boot time from \file{rc.inet2}, or a daemon managed by
\prog{inetd}.  In the latter case you have to have the following entry in
\file{/etc/inetd.conf}:

\begin{tscreen}
\begin{verbatim}
nntp    stream  tcp nowait      news    /usr/etc/in.nntpd    nntpd
\end{verbatim}
\end{tscreen}

If you configure \prog{nntpd} as standalone, make sure that any such
line in \prog{inetd.conf} is commented out. In either case, you have to
make sure there's the following line in \file{/etc/services}:

\begin{tscreen}
\begin{verbatim}
nntp    119/tcp   readnews untp    # Network News Transfer Protocol
\end{verbatim}
\end{tscreen}


To temporarily store any incoming articles, etc, \prog{nntpd} also needs
a \file{.tmp} directory in your news spool. You should create it using

\begin{tscreen}
\# mkdir \newsspool/.tmp\\
\# chown news.news \newsspool/.tmp
\end{tscreen}



\section{Restricting NNTP Access}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{nntp.access}
\index{NNTP!restricting access}
\index{nntpaccess@\file{nntp\_access}}
\index{access!NNTP}

Access to NNTP resources is governed by the file \file{nntp\_access} in
\newslib. Lines in the file describe the access rights granted to
foreign hosts.  Each line has the following format:

\begin{tscreen}
 {\sl site}   read|xfer|both|no    post|no      [!{\sl exceptgroups}]
\end{tscreen}

If a client connects to the NNTP port, \prog{nntpd} attempts to obtain the
host's fully qualified domain name from its IP~address by reverse lookup.
The client's hostname and IP~address are checked against the \var{site}
field of each entry in the order in which they appear in the file.
Matches may be either partial or exact. If an entry matches exactly, it
applies; if the match is partial, it only applies if there is no other
match following which is at least as good.  \var{site} may be specified
in one of the following ways:

\begin{dispitems}
\ditem[\var{hostname}]
        This is a fully qualified domain name of a host. If this matches
        the client's canonical hostname literally, the entry applies,
        and all following entries are ignored.

\ditem[\var{IP address}]
        This is an IP~address in dotted quad notation.  If the client's
        IP~address matches this, the entry applies, and all following
        entries are ignored.

\ditem[\var{domain name}]
        This is a domain name, specified as \host{*.}\var{domain}. If
        the client's hostname matches the domain name, the entry
        matches.

\ditem[\var{network name}]
        This is the name of a network as specified in
        \file{/etc/networks}.  If the network number of the client's
        IP~address matches the network number associated with the
        network name, the entry matches.

%\ditem[\var{subnet name}]
%        This is the name of a subnet of the local network as specified in
%        the \file{/etc/networks} file. The subnet mask is obtained by querying
%        the network interfaces.
%        This option is only enabled if {\tt -DSUBNET} has been defined as
%        compile-time option.
%
%        If the network number of the client's IP~address matches the
%        network number associated with the network name, the entry matches.

\ditem[\keyword{default}]
        The \keyword{default} matches any client.

\end{dispitems}

Entries with a more general site specification should be specified earlier,
because any matches by these will be overridden by later, more exact
matches.


The second and third field describe the access rights granted to the
client. The second details the permissions to retrieve news by pulling
(\keyword{read}), and transmit news by pushing (\keyword{xfer}).  A
value of \keyword{both} enables both, \keyword{no} denies access
altogether. The third field grants the client the right to post
articles, that is, deliver articles with incomplete header information
which is completed by the news software.  If the second field contains
\keyword{no}, the third field is ignored.


The fourth field is optional, and contains a comma-separated list of
groups the client is denied access to.

A sample \file{nntp\_access} file is shown below:

\begin{tscreen}
\begin{verbatim}
#
# by default, anyone may transfer news, but not read or post
default                 xfer            no
#
# public.vbrew.com offers public access via modem, we allow
# them to read and post to any but the local.* groups
public.vbrew.com        read            post    !local
#
# all other hosts at the brewery may read and post
*.vbrew.com             read            post
\end{verbatim}
\end{tscreen}


\section{NNTP Authorization}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{nntp.authorize}
\index{NNTP!authorization}
\index{NNTP!restricting access}
\index{access!NNTP}
\index{authorization!with NNTP}

When capitalizing the access tokens like \keyword{xfer} or
\keyword{read} in the \file{nntp\_acces} file, \prog{nntpd} requires the
authorization from the client for the respective operations. For
instance, when specifying a permission of \keyword{Xfer} or
\keyword{XFER}, \prog{nntpd} will not let the client transfer articles
to your site unless it passes authorization.


The authorization procedure is implemented by means of a new NNTP
command named \keyword{AUTHINFO}. Using this command, the client
transmits a user name and a password to the NNTP server.  \prog{nntpd}
will validate them by checking them against the \file{/etc/passwd}
database, and verify that the user belongs to the \user{nntp} group.


The current implementation of NNTP authorization is only experimental,
and has therefore not been implemented very portably.  The result of
this is that it works only with plain-style password databases; shadow
passwords will not be recognized.



\section{\prog{nntpd} Interaction with C~News}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{nntp.interact}
\index{C News!NNTP support}
\index{NNTP!and C News}

When receiving an article, \prog{nntpd} has to deliver it to the news
subsystem. Depending on whether it was received as a result of an
\keyword{IHAVE} or \keyword{POST} command, the article is handed to
\prog{rnews} or \prog{inews}, respectively. Instead of invoking
\prog{rnews}, you may also configure it (at compile time) to batch the
incoming articles and move the resulting batches to
\newsspool\file{/in.coming}, where they are left for \prog{relaynews} to
pick them up at the next queue run.


To be able to properly perform the ihave/sendme protocol, \prog{nntpd}
has to be able to access the \file{history} file. At compile time, you
therefore have to make sure the path is set correctly.  You should also
make sure that C~news and \prog{nntpd} agree on the format of your
history file. C~news uses \file{dbm} hashing functions to access it;
however, there are quite a number of different and slightly incompatible
implementations of the \file{dbm} library.  If C~news has been linked
with the a different \file{dbm} library than you have in your standard
\file{libc}, you have to link \prog{nntpd} with this library, too.


\index{checking!NNTP}
A typical symptom of \prog{nntpd} and C~news disagreeing on the database
format are error messages in the system log that \prog{nntpd} could not
open it properly, or duplicate articles received via NNTP.  A good test
is to pick an article from your spool area, telnet to the \keyword{nntp}
port, and offer it to \prog{nntpd} as shown in the example below (your
input is marked \var{like this}).  Of course, you have to replace
\var{$<$msg@id$>$} with the message-ID of the article you want to feed
to \prog{nntpd} again.


\begin{tscreen}\small\tt
\$ \var{telnet localhost nntp}		\\
Trying 127.0.0.1...			\\
Connected to loalhost			\\
Escape characters is '\car ]'.		\\
201 vstout NNTP[auth] server version 1.5.11t (16 November 1991) ready at
Sun Feb 6 16:02:32 1194 (no posting)	\\
\var{IHAVE $<$msg@id$>$}		\\
435 Got it.				\\
\var{QUIT}
\end{tscreen}


This conversation shows the proper reaction of \prog{nntpd}; the message
``\term{Got it}'' tells you that it already has this article. If you get
a message of ``\term{335 Ok}'' instead, the lookup in the history file
failed for some reason. Terminate the conversation by typing Ctrl-D.
You can check what has gone wrong by checking the system log;
\prog{nntpd} logs all kinds of messages to the \keyword{daemon} facility
of \prog{syslog}.  An incompatible \file{dbm} library usually manifests
itself in a message complaining that \func{dbminit} failed.

\index{configuring!NNTP|)}
